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DETAILED ACTION 
Response to Amendment 

1 . This office action is responsive to applicant's amendments and remarks received 
on April 11, 2007. Claims 1-6 are currently pending. 



Priority 

2. The foreign priority claim filed March 31 , 2003 was not entered because the 
foreign priority claim was not filed during the time period set forth under 35 U.S.C. 1 19 
(a)-(d) or (f). Applicant may claim benefit of an earlier filing date of a foreign application 
under 35 U.S.C 1 19 (a)-(d) or (f) provided the application is filed within 12 months from 
the earliest date on which such foreign application was filed. 

It is noted that applicant has not addressed the foreign priority matter advanced 
by the Examiner in paragraph 1 of the previous Office Action. Foreign priority continues 
to be denied. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
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A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



3. Claims 1, 2, 4, and 5 rejected under 35 U.S.C. 102(e) as being anticipated by 
Kuno (US 5,802,494). 



Regarding claim 1, Kuno discloses an image transmission system for a mobile robot, 
comprising: 



a camera for capturing an image as an image signal (Figure 2B, 31a and 31b, "video 
cameras"). 



a microphone for capturing sound as a sound signal ("The robot 5 has a video camera 
in its head, a microphone" , column 3, lines 40-41 ) 



human detecting means for detecting a human from the captured sound ("and his or her 
speech is recognized by. the speech recognition device built in the robot 5", Kuno, 
column 24, line 54). The robot in the Kuno patent has a built-in speech recognition 
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system that recognizes the subject is human when speech recognition takes place. In 
other words, when speech recognition takes place, the robot can be assured of the 
humanness of the subject. 

a power drive unit for moving the robot toward the detected human ("Another drive 
mechanism is incorporated in the trunk of the robot 5. When this mechanism is 
actuated, the robot 5 moves in any direction on the floor" , column 28, lines 27-29"; 

Upon receipt of the subject's approval automatically sends an aural, visual, or 

tactile messages to the subject, informing the subject that his or her image is about to 
be transmitted to the monitor section", column 25, line 44; "The robot 5 can move its 
arms and hands, touching the bed or the subject, thus giving a tactual message asking 
the subject if he or she agrees to be monitored ", column 24, line 6); 

an image cut out means for cutting out an image of the detected human according to 
information from the camera; ("the image of the subject's head is extracted from the 
input image (Figure 11 A)", column 9, lines 43-44) ; and 

image transmitting means for transmitting the cut out human image to an external 
terminal ("and the signals showing the subject's image are transmitted to a CRT display 
installed in a monitor room", column 1, lines 60-62) 
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« 

Regarding claim 2, Kuno discloses an image transmission system according to claim 
1 , wherein the system is adapted to detect a moving object from the image signal 
obtained from the camera, and determine that the object is a human from color 
information of the moving object (Figure 19B, i5 "Does the imput image include an 
image of a moving object?"; "the pixels of skin color and hair color are extracted from 
the pixels of other colors", column 8, lines 20-22). 

Regarding claim 4, Kuno discloses an image transmission system according to claim 
1 , further comprising means for monitoring state variables including a current position of 
the robot; the image transmitting means transmitting the monitored state variables in 
addition to the cut out human image (FIG. 28 is a diagram illustrating how to determine the 
position of the robot, column 3, lines 16-17, "The physician looks at the subject's face 
being displayed on the display of the monitor console and also checks the physical 
conditions being displayed on the other displays of the console, in order to decide what 
he or she should do for the subject.", column 5, lines 26-30, "the circuit 40 starts 
transmitting the image data representing the image of the subject, to the monitor section 
2", column 5, lines 14-16, "Meanwhile, the electronics sensors 33 installed in the 
sickroom, such as the hemadynamometer and the electrocardiograph, both attached to 
the subject, output diagnosis signals, representing the physical conditions of the 
subject. The video signals and the diagnosis signals are input to the signal processor 32 
incorporated in the data-processing/control section 4. The processor 32 processes 
these input signals, thereby generating image data and diagnosis data. The image data 
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and the diagnosis data are supplied to the abnormality decision circuit 34 incorporated 
in the robot 5.", column 5, lines 10-20) 



Regarding claim 5, Kuno discloses an image transmission system according to claim 
1 , wherein the system is adapted to have the robot direct the camera toward the 
position of the detected human ("Since the video camera 31a built in the robot 5 is 
directed to only the subject on the bed", column 7, lines 30-31). 



Claim Rejections - 35 USC § 103 

Claims 1. 2. 5. and 6 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a 
person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness 
under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 
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5. Claims 1, 2, 5 and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over the combination of Higaki et. al. (US 2004/0028260 A1) and Kuno (5,802,494). 

In Regards to Claim 1 : 

Higaki discloses an image transmission system for a mobile robot, comprising: 
a camera for capturing an image as an image signal ("employing 2-color CCDs, with L 
and R denoting the left side camera and the right side camera respectively", paragraph 
41; Figure 1, 1L and 1R). 

a microphone for capturing sound as a sound signal ("reference symbol 21 denotes a 
microphone that picks up the voice of a person speaking", paragraph 41; Figure 1, 21) 

human detecting means for detecting a human from the captured sound ("voice 
recognition section which recognizes the content of the utterance picked up by the 
microphone", paragraph 41). 

a power drive unit for moving the robot toward the detected human ("a drive control 
section", paragraph 41; Figure 1, numeral 9); 

an image cut out means for cutting out an image of the detected human 
according to information from the camera ("reference symbol 54 denotes an outline 
extraction section which extracts an outline"; Figure 1 , 54) ; and 
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Higaki does not disclose an image transmitting means for transmitting a human image 
to an external terminal. 

However, Kuno (5,802,494) does disclose an image transmitting means for transmitting 
a human image to an external terminal ("and the signals showing the subject's image 
are transmitted to a CRT display installed in a monitor room", column 1 , lines 60-62, 
"the image of the subject's head is extracted from the input image (Figure 11 A)", column 
9, lines 43-44) 

It would have been obvious at the time of the invention to a person of ordinary skill in 
the art to combine the image transmission system for a mobile robot with the elements 
as disclosed by Higaki with an image transmitting means for transmitting a human 
image to an external terminal taught by Kuno in order to monitor an individual from a 
remote location and thereby increase the chance of finding a lost child. ("The CRT 
display displays the image of the subject, whereby a physician in the monitor room can 
observe the subject", column 1 , lines 22-24) 

In Regards to Claim 2: 

Higaki teaches an image transmission system according to claim 1, wherein the system 
is adapted to detect a moving object from the image signal obtained from the camera 
("a moving object ID at 116", paragraph 87 and determine that the object is a human 
from color information of the moving object ("make an area inside the outline having a 
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predetermined color a candidate for a hand of a person", paragraph 17, "denotes a face 
recognition section which recognizes a person's face from the color image, the skin- 
color area image", paragraph 19) 

In Regards to Claim 5: 

Higaki discloses an image transmission system according to claim 1, wherein the 
system is adapted to have the robot direct the camera toward the position of the 
detected human ("obtains the pan angle and tilt angle of the cameras 1L and 1R (step 
S81 ). The line of sight instruction section sends the obtained pan angle and the tilt 
angle to the action control section 9 (step S82). As a result, the cameras 1L and 1R 
always come to face the direction of the head of the person who issues the instruction 
"come', it becomes possible to track the person", paragraph 94) 

In Regards to Claim 6: 

Higaki discloses an image transmission system according to claim 1, wherein the 
system further comprises means for measuring a distance to the detected human 
according to the information from the camera, and providing a target of a movement to 
said mobile robot ("a distance calculation device that calculates a distance to the body 
being the candidate, from distance information of each pixel within the outline in the 
image", paragraph 12 and "the movement instruction section 64, based on the moving 
object data 90, tracks the person who instructed "come" (step S95)", paragraph 95). 
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Claim 3 

6. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Higaki et. al. (US 2004/0028260 A1) and Kuno (5,802,494) as applied to 
claim 1 above, and further in view of Shinichi (2000-326274). 

In Regards to Claim 3: 

While the Higaki and Kuno combination discloses a microphone as described above, 
the Higaki and Kuno combination does not teach determining a direction of a sound 
source from the sound signal obtained from the microphone. 

However, Shinichi does teach a system, in the same problem solving area of locating a 
source of sound that detects the direction of a sound source ("direction of a sound 
source is detected", paragraph 5). 

It would have been obvious at the time of the invention to a person of ordinary skill in 
the art to combine the image transmission system for a mobile robot with the elements 
as disclosed by Higaki and Kuno in the claim 1 discussion above with a system as 
taught by Shinichi that is adapted to determine a direction of a sound source from the 
sound signal obtained from the microphone because this directional information 
provided by the sound signal allows the robot an additional way to be able to locate the 
person ("turned the robot in this direction" paragraph 5 of Shinichi and "person retrieval 
processing" paragraph 5 of Shinichi). 
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Claim 4 



7. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Higaki et. al. (US 2004/0028260 A1) and Kuno (5,467,403) as applied to 
claim 1 above, and further in view of Ishii (6,278,904). 

In Regards to Claim 4: 

Higaki, as part of the Higaki and Kuno combination applied to claim 1 above, teaches 
means for monitoring state variables ("the person information map 10 defines the 
relative position between the person and the autonomous relative position between the 
person and the autonomous robot R, in order to grasp where the person issuing 
instructions currently is. The person information map 110 includes: an individual person 
ID 1 1 1 for uniquely specifying persons, a relative position to self 1 12, a face object ID 
113, and individual person ID 114, a posture ID 115, a moving object ID 116, a face 
position coordinate 1 18 in the real space, a hand position coordinate 1 19, a distance to 
self 120, a relative angle to self 121, an outline contact point coordinate 122, and a 
head vertex point coordinate 123", paragraph 87, "the operation for renewing the person 
information map in the object integration section 62 is described with reference to 
Figure 15", paragraph 88 and "the operation for renewing the person information map in 
the object integration section 62 is described with reference to Figure 15", paragraph 
88). 
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Higaki does not disclose an image transmitting means transmitting the monitored state 
variable including current position of the robot. 

Ishii does disclose an image transmitting means transmitting the monitored state 
variables including the current position of the robot ("In the first embodiment, the 
information captured through the image sensor 1 1 and the audio sensor 12 is used for 
the purpose of detecting a current position of the robot 10 in order for the robot 10 to 
move around all objects to be monitored or to monitor a specified object. The 
information captured through the image sensor 1 1 and the audio sensor 12 is also 
stored in the robot 10, or transferred externally through the communications device 19 
and stored in an external device, as the monitoring data", column 5, lines 10-15). 

It would have been obvious at the time of the invention to a person of ordinary skill in 
the art to combine the image transmission system for a mobile robot with the elements 
as disclosed by Higaki and Kuno in the claim 1 discussion above with an image 
transmitting means transmitting the monitored state variables including current position 
of the robot as taught by Ishii, in order to be able to observe a predetermined object and 
keep track of its location and other state variables of interest ("detecting a current 
position of the robot 10 in order for the robot to move around all objects to be monitored 
or to monitor a specified object" and "the information captured through the image sensor 
1 1 and the audio sensor 12 is also stored in the robot 10, or transferred externally 
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through the communications device 19 and stored in an external device, as the 
monitoring data", Ishii, column 5, lines 10-15). 



Response to Arguments 

Summary of Applicant's Remarks: The Kuno patent fails to disclose a "human detecting 
means for detecting a human from the captured sound' at response page 6, top 
paragraph. 

Examiner's Response : The following excerpt is provided from the Kuno patent in 
column 24, starting at line 54: "and his or speech is recognized by the speech 
recognition device built in the robot 5, whereby the subjects approval or disapproval is 
conveyed to the physician". By "approval" and "disapproval", the subject provides 
permission for being monitored. Thus, the robot in the Kuno patent has a built-in 
speech recognition system that recognizes the subject is human when speech 
recognition takes place. In other words, when speech recognition takes place, the robot 
can be assured of the humanness of the subject. 

Summary of Applicant's Remarks: The Kuno patent fails to disclose providing "a power 
drive unit for moving the robot toward the detected human" at response page 6, top 
paragraph. 
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Examiner's Response: The following excerpt is provided from the Kuno patent in 
column 28 starting at line 28: "Another drive mechanism is incorporated in the trunk of 
the robot 5. When this mechanism is actuated, the robot 5 moves in any direction on the 
floor. The drive mechanisms incorporated in the robot 5 are remote-controlled by 
operating the monitor console of the monitor section 2. Thus, under the control of the 
monitor section 2, the robot 5 is moved to a desired position with respect to the subject, 
and its head is turned, bent and vertically moved until its nose (i.e., the zoom-lens 
cylinder) is directed to the subject's face." 

Thus, the robot of Kuno has a power drive unit that is responsible for moving the 
robot. Referring to the aforementioned speech recognition of Kuno, Once the robot has 
recognized aural "approval," the robot then uses the power drive unit to move its hand 

towards the patient ("Upon receipt of the subject's approval automatically sends an 

aural, visual, or tactile messages to the subject, informing the subject that his or her 
image is about to be transmitted to the monitor section", column 25, line 44; "The robot 
5 can move its arms and hands, touching the bed or the subject, thus giving a tactual 
message asking the subject if he or she agrees to be monitored.", column 24, line 6) to 
communicate to the patient that his or her image will now be transmitted to the 
physician in the console room. 
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Summary of Applicant's Remarks: Kuno patent fails to disclose "an image cut out 
means for cutting out an image of the detected human according to information from the 
camera" at response page 6, second paragraph. 

Examiner's Response: the following excerpt is provided from the Kuno patent in column 
9, lines 43 to 44: "the image of the subject's head is extracted from the input image 
(Figure 11 A)." 

More specifically, the Kuno patent recites the following means of extraction in 
column 9 starting at line 21: "The pixels extracted from the input image shown in FIG. 
1 1A and defining the image shown in FIG. 1 1B are labeled in step c2. Then, in step c3, 
the video signals representing the labeled pixels are processed, whereby data items are 
obtained which define rectangles. Each of these rectangles circumscribes a groups of 
the labeled pixels which are close to one another. Of these rectangles, those which are 
so small to be regarded as resulting from noise, and those which are long enough to be 
regarded as representing a crease of the bedding are deleted from the image, thereby 
forming the image shown in FIG. 11C. Then, in step c4, data is obtained which 
represents a rectangle circumscribing the rectangles shown in FIG. 11C, as is illustrated 
in FIG. 1 1D. In step c5, it is determined whether or not the rectangle is large enough to 
represent the subject's head. If YES, the rectangle is considered to represent the 
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subject's head. If NO, the rectangular area is expanded so that it can be understood to 
represent the subject's head. Thus, the image of the subject's head is extracted from 
the input image (FIG. 1 1 A)." 

Summary of Applicant's Remarks: Kuno patent fails to disclose a robot that approaches 
"the subject to face him and take a picture" at response page 6, penultimate paragraph. 

Examiner's Response : The following excerpt is provided from the Kuno patent in 
column 28, line 33: "Thus, under the control of the monitor section 2, the robot 5 is 
moved to a desired position with respect to the subject, and its head is turned, bent and 
vertically moved until its nose (i.e., the zoom-lens cylinder) is directed to the subject's 
face." 

Summary of Applicant's Remarks: Applicant claims that "Higaki is not a proper 
reference in these rejections, because it barred by 35 U.S.C. 103(c). Higaki is available 
as a reference only under 35 U.S.C. 102(e), if at all. Higaki has filing date of August 3, 
2003 and published on February 12, 2004. In contrast, the present application was filed 
April 1, 2004" at response page 7, bottom paragraph. 

Examiner's Response : "35 U.S.C. 103(c), as amended by the CREATE Act, applies 
only to subject matter which qualifies as prior art under 35 U.S.C. 102(e), (f), or (g), and 
which is being relied upon in a rejection under 35 U.S.C. 103.", MPEP 2146. Higaki is 
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prior art under 102(a). Higaki was published on February 12, 2004, which is before 
applicant's filing date of April 1, 2004. Therefore, Higaki is not barred under 103(c). 



Summary of Applicant's Remarks: Applicant states that claim 3 is not taught by the 
combination of Higaki, Kuno and Shinichi because "Higaki may not be used as prior art 
against the present application" at response page 9, third paragraph. 

Examiner's Response : However, for the same reason set forth directly above, Higaki is 
not barred under 103 (c) and therefore, this argument is moot. 

Summary of Applicant's Remarks: Applicant claims that Shinichi does not teach a 
"human detecting means for detecting a human from the captured sound" and a "power 
drive unit for moving the robot toward the detected human" at response page 10, 
second paragraph. 

Examiner's Response : Examiner's rejection of claim 3 does not rely on Shinichi 
teaching either the "human detecting means for detecting a human from the captured 
sound" or the "power drive unit for moving the robot toward the detected human." 
These features are taught by the Kuno patent as discussed above. 



Application/Control Number: 10/814,343 
Art Unit: 2621 



Page 18 



Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Elisa M. Rice whose telephone number is (571 )270- 
1580. The examiner can normally be reached on 8:00a.m.-5:30p.m. EST Monday thru 
Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian P. Werner can be reached on (571)272-7401. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
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Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Elisa M. Rice 

Assistant Patent Examiner 
2609 

EMR 



/ Brian P. Werner/ 

Supervisory Patent Examiner (SPE), Art Unit 2624 



